Inference in Binary Pair-wise Markov Random Fields through Self-Avoiding Walks

The algorithms for finding a Maximum A-Posteriori (MAP) assignment or marginal distribution in a pairwise Markov Random Field (MRF) have been of great recent interest due to their wide application in the context of vision, coding, communication and discrete optimization problems. The max-product (MP) and (sum-product) belief-propagation (BP) algorithm and their variants (e.g. tree re-weighted (TRW) algorithm) have received much recent attention due to their simplicity and effectiveness in the context of MRFs with relatively simpler structure. Despite exciting progress however, the questions of correctness, convergence, approximation guarantees and possible improvements for these algorithms remain largely unresolved. As a main result of this paper, we obtain the following surprising equivalence for any binary pair-wise MRF on a graph G: the max-marginal (resp. marginal probability) of a node, say v, in G is the same as the max-marginal (resp. marg. prob.) of the root node of the tree MRF obtained by performing a self-avoiding walk1 on G starting from v. This result can be seen as a generalization of a recent result of D. Weitz [31] for hard-core model for computing marginal probability. We obtain three very important implications of this result. First, we obtain corrections of the MP and BP algorithms in the presence of cycles to obtain exact MAP or marg. prob. by means of a message-passing algorithm unlike the complicated centralized junction-tree algorithm. Second, the algorithm for exact computation suggests a natural heuristic which by means of representative experimental results is shown to be more effective than the TRW algorithm. Third, we obtain analytic error bounds on the performance of BP for graphs with cycles in terms of structural properties of the MRF G. We expect that our self-avoiding walk based approach will have many more implications beyond those described in this paper.